FRVT Face Mask Effects

Status

[last updated: March 4, 2021]

NIST has published an update to NISTIR 8331 - Ongoing FRVT Part 6B: Face recognition accuracy with face masks using post-COVID-19 algorithms aimed at quantifying face recognition accuracy for people wearing masks. This report adds 35 new algorithms submitted to FRVT 1:1 since the last report (and includes cumulative results for 198 algorithms evaluated to date).

Results

[last updated: July 14 , 2021]

The accuracy table currently shows the top performing 1:1 algorithms evaluated on masked images. Results are tabulated on the VISABORDER dataset where the probe image is masked (and the enrollment image remains unmasked). It also includes baseline FNMR when both images are not masked. FNMR values are stated at a fixed threshold calibrated to give FMR = 0.00001 on unmasked images. FNMR is the proportion of mated comparisons below a threshold set to achieve the false match rate (FMR) specified. FMR is the proportion of impostor comparisons at or above that threshold.

The results table will be updated on a monthly basis as algorithms and computations complete, as datasets are added, and as new analyses are developed.

Reports

All Ongoing FRVT Face Mask Effects reports can be accessed from here.

Motivation

Traditionally, face recognition systems (in cooperative settings) are presented with mostly non-occluded faces, which include primary facial features such as the eyes, nose, and mouth. However, there are a number of circumstances in which faces are occluded by masks such as in pandemics, medical settings, excessive pollution, or laboratories. As of late, the widespread requirement that people wear face masks in public places has driven a need to understand how cooperative face recognition technology deals with occluded faces, often with just the periocular area and above visible. There has been an increasing number of providers that have developed “face mask capable” face recognition capabilities, and therefore, has driven the need to quantify the performance of modern face recognition on masked faces.

How to Participate

Currently, all algorithms submitted to the Ongoing FRVT 1:1 Verification test will be evaluated against both unmasked and masked dataset(s).

The idea is that developers should submit an integrated package that:

  • On an unmasked image, extracts features from both the full face and the periocular region
  • On a masked image, extracts features from the periocular region
  • Compares any combination of unmasked and masked faces

At least one developer has done this recently, with FMR approximately constant for any combination.

The test is designed this way to mimic operational reality: some images will have masks, some will not (especially enrollment samples from a database or ID card).

Please refer to the FRVT 1:1 webpage for information on how to participate.

Contact Information

Inquiries and comments may be submitted to frvt@nist.gov.

Subscribe to the FRVT mailing list to receive emails when announcements or updates are made.